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CLAIMS 

1. An image composition system for compositing a real 
image in a line-of-sight direction of a user with another 
image, comprising: 

5 a display unit which is wearable on a head of the user, 

and displays a composite image; 

a position sensor for detecting the line-of-sight 
direction of the user, and outputting line-of-sight 
information; 

10 a determination unit for determining a display region 

where the other image is to be displayed, in accordance with 

the line-of-sight information; and 

a composition unit for compositing the other image 

on the determined display region, 
15 wherein the other image is used to display 

information that helps operations of the user. 

2. The system according to claim 1, wherein said display 
unit has an optical see-through structure, and the user can 
observe a real space via said display unit. 

20 3. The system according to claim 1, further comprising: 
a first image taking device for obtaining a video of 
a real space observed from a viewpoint of the user, and 

wherein said composition unit displays the video 
obtained by said first image taking device on said display 
25 unit, and superimposes the other image on the display region 
determined by said determination unit. 
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4. The system according to claim 1^ wherein the other 
image information is a video obtained by a second image 
taking device for taking an image from a viewpoint other 
than a viewpoint of the user. 
5 5. The system according to claim 1, wherein the other 
image information is text information. 

6. The system according to claim 5, further comprising: 
a memory for holding a pair of the text information 

and time information indicating a display timing of the text 
10 information, and 

wherein said composition unit switches the text 
information to be displayed on the display region in 
accordance with the time information held by said memory. 

7. The system according to claim 1, wherein said 
15 determination unit comprises: 

a setting unit for setting a space region for 
displaying the other image in the real space; and 

a conversion unit for converting the space region set 
by said setting unit into the display region on said display 
20 unit on the basis of a position and posture of the user. 

8. The system according to claim 1, further comprising: 
a gesture detection unit capable of detecting, a 

predetermined action of the user, and 

wherein said composition unit controls to turn on/off 
25 display of the other image in response to a predetermined 
action detected by said gesture detection unit. 
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9. The system according to claim 8, wherein said 
composition unit switches contents of the other image to 
be displayed on the display region in response to a 
predetermined action detected by said gesture detection 

5 unit. 

10. The system according to claim 1, wherein the 
information that helps the operations of the user is dialog 
information. 

11. The system according to claim 1, wherein the 

10 information that helps the operations of the user is an 
image obtained by taking an image of an action of the user. 

12. An information processing method of displaying a 
composite image of a real image in a line~of-sight direction 
of a user and another image on a display unit which is 

15 wearable on a head of the user, comprising the steps of: 
detecting the line-of-sight direction of the user to 
acquire line-of -sight information; 

determining a display region where the other image 
is to be displayed, in accordance with the line-of-sight 
20 information; and 

compositing the other image on the determined display 
region, 

wherein the other image is used to display 
information that helps operations of the user. 
25 13. The method according to claim 12, wherein the other 
image information is a video obtained by image taking means 
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for taking an image from a viewpoint other than a viewpoint 
of the user. 

14, The method according to claim 12^ wherein the 
information that helps the operations of the user is word 
information. 

15. A computer readable medium that stores a program for 
making a computer execute an information processing method 
of displaying a composite image of a real image in a 
line-of-sight direction of a user and another image on a 
display unit which is wearable on a head of the user, said 
control program making the computer execute a process for: 

detecting the line-of -sight direction of the user to 

acquire line-of-sight informations- 
determining a display region where the other image 

is to be displayed, in accordance with the line-of-sight 

information; and 

compositing the other image on the determined display 

region. 



